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“Built-In” Action/Issues Tracking and Post-Ops Analysis Tool for Realtime 
Console Operations 

Marshall Space Flight Center’s (MSFC) Payload Operations Integration Center 
(POIC) for the International Space Station (ISS) uses a number of formal 
databases to manage and track flight plan changes, onboard and ground 
equipment anomalies, and other events. However, individual console positions 
encounter many action items and/or occurrences that don’t fit neatly into the 
databases, and while console logs are comprehensive, manual or automated 
searches do not always yield consistent results. The Payload Communications 
Manager (PAYCOM) team, whose members speak directly with the ISS onboard 
crew with respect to NASA payload operations, has found a creative way to 
reformat a mandatory Daily Report to organize action items, standing reminders, 
significant events, and other comments. While the report keeps others appraised 
of PAYCOMs activities and issues of the moment, the format makes it easy to 
capture very brief summaries of the items in a “Roll Off Matrix”, including start 
and stop dates, resolution, and possible applicability to future ops. The matrix 
provides accountability for all action items, gives direct insight into the issues 
surrounding various payloads and methods of dealing with them, yields indirect 
information on PAYCOM priorities and processes, and provides a roadmap that 
makes it easier to get back to extensive details if needed. This paper describes 
how the ISS PAYCOM Daily Report and Roll Off Matrix are organized, used, and 
inter-related to each other and the PAYCOM operations log. While the 
application is for a manned vehicle, the concepts could apply in a wide spectrum 
of operational settings. 
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Abstract - At Marshall Space Flight Center’s (MSFC) 
Payload Operations Integration Center (POIC) for the 
International Space Station (ISS), the Payload 
Communications Manager (PAYCOM) team, whose 
members speak directly with the ISS onboard crew with 
respect to NASA payload operations, has found a creative 
way to reformat a mandatory Daily Report to organize 
action items, standing reminders, significant events, and 
other comments. While the report keeps others apprised of 
PAYCOM’s current activities and issues, very brief 
summaries of the items are put into a “Roll Off Matrix”, 
including start and stop dates, resolution, and possible 
applicability to future ops. The matrix provides 
accountability for all action items, gives direct insight into 
issues regarding payloads, control center operations, and 
methods, yields indirect information on PAYCOM priorities 
and processes, and provides a roadmap for locating 
extensive details if needed. This paper describes how the 
Daily Report and Roll Off Matrix are organized, used, and 
inter-related to each other and the PAYCOM operations log. 
While the application is for a manned vehicle, the concepts 
could apply in a wide spectrum of operational settings. 
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Introduction 

Operational log-keeping has been practiced for millennia, A 
log can be a powerful tool for maintaining continuity 
between shifts, reconstructing events, resolving anomalies, 
and distilling better ways of doing things. Early spaceflight 
programs relied on handwritten logs, and even though 
console operators developed unique symbols or shorthand, 
the pace of operations was such that operators could only jot 
down the bare essentials of what they observed. Because 
logs were handwritten, searching for either specifics or 
trends was very time-consuming. 


Today, inexpensive computers, office automation software, 
and “instant-replay” voice communications archive systems 
allow console operators to prepare extremely detailed logs 
by the end of their shifts. This is obviously helpful if event 
reconstruction is needed to analyze a specific incident 
whose date(s) is/are known, and for assessment of a 
relatively short flight lasting one or two weeks. 


Long-duration missions such as those flown on the 
International Space Station present some new challenges: 

- System or payload activities may span weeks or 
months, and a series of closely related activities may 
span years. 

- Crew members fly for months, and their task loading 
can make it difficult for them to remember nuances and 
“gotchas” between performances. 

- Ground support personnel are often involved with three 
or four ISS increments concurrently - while one is 
flying, others are in preparation or post-flight. (An 
increment is a complement of payloads and activities 
associated with a given ISS expedition, as well as the 
time frame that the expedition crew is onboard.) 

- The sheer number of log entries related to a given topic 
can make analysis difficult, even if a text search is used 
to reduce the number of entries under consideration. 

- Formal databases document details upon details of 
planning criteria, equipment anomalies, ground team 
anomalies, and so forth, but in terms of ready reference 
to help console operators get through their shift, they 
may provide so much background that the foreground 
goes underground. 

- As international partners begin flying their modules, 
systems, payloads, and flight crews, handshakes across 
different partners’ hardware, software, and organizations 
will become more complex, no doubt leading to more 
idiosyncrasies and “unadvertised features”. 
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This paper is written from the ISS Payload Communicator 
(PAYCOM) perspective. PAYCOMs are responsible for 
ISS crew communication for NASA payloads, which 
includes: 

- Space-to-Ground (S/G) communication with crew for 
NASA payload issues 

- Preparing written material for the Daily Summary 
based on ground team inputs 

- Managing voice and/or video conferences between the 
crew and payload ground teams 

- Reviewing procedures and operations documents with 
an emphasis on crew issues 

- Tracking crew questions and/or requests related to 
NASA payloads. Some of these are handed off to other 
positions based on intra-team coordination, but 
PAYCOM is the default actionee. 

The PAYCOM console is nominally staffed during crew 
awake hours on normal workdays. This is typically 16 hours 
a day, 5 days a week, and the gaps in continuous coverage 
accentuate the need for good handover and “cheat sheet” 
materials. 


Body 

Daily Report and Rolloff Matrix Evolution 

In the course of operating since 2001, PAYCOMs tried 
several methods for keeping up with questions and actions. 
Items that could be resolved within 1-3 days could usually 
be managed via handover entries in the console log, but 
longer-term items tended to “drop through the cracks”. The 
“Tasks” feature of the email system (Microsoft Outlook) 
was used for a while, but problems included subfolder 
management, disappearance of items due to inadvertent 
mouse clicks on checkboxes, frustration/wasted motion due 
to nesting of messages and enclosures, and how easy it was 
to forget to check the system, especially on hectic days. 
Filing schemes for email messages outside of the “Tasks” 
structure yielded similar results. 

At a PAYCOM meeting in early 2007, someone noted that 
all of the methods attempted were internal to the PAYCOM 
team - no other console position had insight into what we 
were tracking unless they’d made a note of it themselves, 
and their notes might not agree with ours. 

It occurred to us that a Daily Report (DR) that we’re 
required to submit (on any day that the console is staffed) to 
the Payload Operations Director (POD) and the Payload 
Operations Manager (POM) had a place for “Forward 
Actions”, but the section was not well-defined. Other 


sections had been defined early in the ISS program, but 
hadn’t been validated and/or updated with respect to current 
operations. The scheme that evolved has these features: 

- Since the DR is mandatory, we can guarantee that 
PAYCOM will open and look at it every day on console. 

- New DRs are created via a “Save As” of the previous 
report. This eliminates errors from cutting and pasting 
from one console log entry to another. 

- New or modified information is entered in blue text, 
and changed to black in the following report. Entries 
being closed are put back into blue text for their last 
appearance in the report. Authors and editors include 
date/initials on revisions. 

- Separate sections exist for Actions (discrete tasks, 
usually of a one-time nature) and Reminders (tasks or 
things to remember tied to recurring events or 
circumstances, e.g. “gotchas”). 

- The section originally titled Malfunctions and 
Recoveries now includes all Significant Events, 
including notable successes. Sections covering info that 
“belongs” to other teams have been eliminated. 

- POD/POM are now aware of what we think is 
important, and can ask us to 1) delete items they’re 
working and/or 2) add items they’d like us to take on. 
“Trim the overlaps, fill in the underlaps.” 

These modifications helped a great deal for day-to-day ops, 
but the question arose, “If someone asked us to account for 
all of our unique action items (e.g., those not covered in 
formal databases), could we summarize what they are/were, 
and how they were resolved?” Well, we could . . . but 
digging through all the DRs would take an incredibly long 
time, and one would have to sort through repetitive entries 
from the date an item appeared to the date it was removed. 
Someone suggested that we “roll off ’ the action items into a 
matrix for future reference, including start and stop dates to 
show the time frame. We concluded that summarizing 
Reminders, Significant Events, and other Comments would 
also be useful. 


Organization and Content 

The DR and Rolloff Matrix formats that emerged are shown 
in Figures 1 and 2, respectively. The Rolloff Matrix 
example shows several but not all types of entries. As of 
October, 2007, the actual matrix covered approximately 1.5 
years of operations and contained about 350 entries. 
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PAYCOM Daily Report - SOP 1.3 

Submit to POD via email within one hour scheduled beginning of crew sleep. This report 
covers hours that me PAYCOM position was staffed 

Start Time (GMT Day/TIme): 271/1 3:30 GMT 

End Time (GMT Day/Time): 271/ 21 :30 GMT 

Calendar Day (DD-MMM-YYYY): 26 Sep 2007 

Submitted By (Operator Name): Dave Scott 

PAYLOAD CREW ACTIVITIES - OSTPV AND TASK USX EXECUTION 

Performed per OSTP (Note any duration delta given by crew or observed, and items 
based on negative reporting) 

• MSG-SAME-STARTUP ST 

• SAME-SAMPLE-CHGOUT 

• SAM E-IM PACT OR-EXC H G 

OSTP - Aborted. Not Performed, or Deleted 


Added to OSTP 


Performed per Task List (marked complete! 
- None 


Available on Task List (not marked) 

- CEO-OPS 


SIGNIFICANT PAYLOAD EVENTS / RESOLUTION (Include ops prep implications) 

(New items in Blue text Usually in DR for just one day) 

Notable Successes 

• SAME Sample C ha n geout a nd {0}pA$gxCh a ngeout ve ry smooth . SAM E PD told US 
Rep everything seems to be working fine — have run another test point, received 

data, and initial analysis seems to confirm one of their hypotheses. 

Malfunctions. Anomalies & Recoveries 


Other 


ACTION ITEMS 

Opened Today (Blue text) 


Pending / In-Work (Show changes in blue text for one day only) 

• Reference OCA message 15-xxxx (wh^ gjgjj(^[ per approved fcolallOOOOl 5) in DS for 
Big Picture words on upgrading ER4 ELC to an A31p. (NWP 193) GMT 198— Still 
waiting on JED I Message to be created and action assigned on OCR. SLJ 

• Give feedback to Clay on results of ANITA trouble shooting once it's working properly. 
During ANITA troubleshooting GMT 264 Clay expressed interest in seeing/knowing the 
results. Passed request to PD, who said see what they could do (gpipj^x -264/21 :58). 
(NWPGMT 265) DS 268 advised Clay ANITA appears to be working OK. Close action 
after we tell him what we think caused the anomaly - PD still thinking about this. 
(Consolidated by Scotty GMT 267). 


Closed Today (Blue text) 

• Track - CEO owes Clay words regarding the quality of the photos he took of his Eastern 
China target on GMT 235. (NWP GMT 236) CEO provided feedback in target list for 
GMT 270. (Closed by DWS GMT 271) 


OTHER COMMENTS - ISSUES, CONCERNS, KUDOS 
(New items in Blue text Usually in DR for just one day) 


REMINDERS 

Added Today (Blue text) 


Current (Alphabetically by payload name. Show changes in blue text for one day only) 

• CEO Targets - Unless otherwise directed that there has been a philosophy change 
gjye the crew a heads up in DPCe when there are targets in their post-sleep. (SOP 
7.13) 

• Any HRF activity requiring laptop - crew in mDPC if powered via UOP or rack 
Originated 2007/089. 

• Stowage - Ask crew for S/N, P/N, and/or Name Labels for items we’re not certain about 
as they’re being handled bv crew whenever possible to avoid *go-backs\ e g, crew 
shows video of or mentions something they found. {Based on EFN F01 7562 related to 
DS 2007/200 Q&A). 

Removed Today (Blue text) 



Note - This is a composite built from several reports for the purpose of illustrating features. 
Since the thrust of this paper is a method and not the actual content, acronyms are not listed. 
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Figure 1 - PAYCOM Daily Report Format 
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1 Facility 


C D E 

I GMT Date I GMT Date I 

Open | Closed [Description 


Samp I e Roli Off M at r I x , x i s 


I Disposition 


During PFM crew conference, PI agreed to info from PI uplinked in message 1 3- 
2006/118 2006/122 provide details of previous PFMI results to crew. 0157. Referenced in DS 123 


I Audited through 2007/271 - dws 12 Oct 2007 


Earth Obs 


A 2006/118 2006/123 


SE 2006/124 2006/124 


A 2006/129 2006/137 


A 2006/131 2006/136 


A 2006/138 


Ask Jeff the S/N and B/C of the touchpad in 
MSG airlock at GMT 123 DPC. 

It appeared that MSG could not command 
recorders. Crew checked wires, all seated; got 
beep sound when trying to press record, tapes 
used; replaced tapes, proceeded OK. 

Question from DPCe for CEO: Some targets in 
world map are hard to find. Any plan to put all 
targets in world map? 

For PFMI on 136, advise Jeff if PFMI Sample 
Removal, PFMI Tape Removal, and MSG 
Shutdown can be done back-to-back. Won't 
know until <24 hours prior. 

Jeffs BCAF setup differs from procedure, but 
PD team likes it, has questions about parts 
used, requests photo. 


Touch pad S/N 01-360 deployed. DR 
123 includes additional info about 4 
MSG video arms. 

Problem caused by connectivity issue at 
MSG TSC. 


Detailed CEO answer submitted for DS 
136. Interesting reading. 

DPCm 1 36 - Told Jeff he can do them 
back to back like last time. (File already 
downlinked.) 

PA7COM sent ejqsla nation to PD on 142, 
photo on 145. Close action when BCAT 
satisfied. 


PDIF 

POIF 

SPHERES 

A 

A 

2006/138 

2006/138 

2006/139 

Actions related to Power Outage GMT 140-141 
Per POD, on Sat morning 140, ask Jeff to be 
sure camera/camcorders in manual mode 
(1 38 DPCe Jeff hadn't checked) 

Crew didn't have good "big picture" objectives. 

Words placed in DS140. 

(Check DRs and logs to verify done.) 

Good sportsmanship all the way around. 

BOSS 

CBOSS-FDI 

C 

2006/184 


found words to set him at ease during run. 

2 anomalies during IP address update. New 

Event was scheduled last-minute due to 
STS scrub. 

PAR HRF2 S/W 9. No residual impact 

HRF 

HRF PC 

SE 

2007/100 


h/w detected, LAN Connection 5 (expected 3 or 
4) 

On Day 2 CCISS Ops, tell FE-2 he needs to 

Could happen again in HD changeout if 
IP address reset 

[Re-classifyas action, or will this hold 

HRF 

CCISS 

R 

2007/176 


physically power off emerald brick. (Skips step 
on Day 1.) 

Ask for S/N, P/N, and/or Name Labels for items 

true for all CCISS ops? ] 

Goal is to prevent go-backs. Based on 

STOWAGE 

STOWAGE 

R 

2007/200 


we're not certain about as theVre beina 
handled bvcrew. 

EFN F017562 related to DS 2007/200 
G&A 

Earth Obs 

CEO 

A 

2007/236 

2007/271 

CEO words to Clay regarding quality of Eastern Feedback in CEO target list GMT 270. 
China photos taken GMT 235. 

Sample Changeout& Impactor Changeout very PD has run another Kapton test point 

MSG 

SAME 

SE 

2007/271 


smooth. Everything seems to be working fine. 

Initial analysis seems to confirm one of 


their hypotheses. 

Note - This is an edited excerpt from the production matrix. 

Since the thrust of this paper is a method and not the actual content, acronyms are not listed. 
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Discussion 

In the DR, content entries appear with bullet symbols under 
the appropriate heading. Visual organization, including 
header and content text color, is important for navigation 
and clarity. New and changed content are in blue text in the 
first report in which they appear. Text is changed to black 
for subsequent reports, then back to blue in the last report in 
which it appears. This makes it easy to see what is changing, 
and the “Opened Today’ and “Closed Today” headings act 
as prompts for a) drafting the next day’s report (with a 
“Save As” command), and b) moving Actions and 
Reminders to the Rolloff Matrix. 

Content in the DR should summarize each topic very 
briefly. Wholesale copying and pasting from detailed 
console log entries tends to clutter things up. Instead, a 
simple reference to “ log DDD/hhhh” (DDD -> Day of year, 
hhh -> GMT 24-hour time) can guide the reader to in-depth 
information. 

Some challenges encountered in preparing DRs include: 

Making sure Actions suggest what the console operator 
needs to do, not merely describe an issue or status. 

Obtaining consistency among operators, e.g., 
establishing what constitutes a Significant Event, when 
an action should be carried in the DR vs. handling it in 
handover notes within the console log. 

Keeping the entries brief. This was mentioned in a 
separate paragraph above. It’s important enough to 
emphasize again . . . and again. 

Following through on protocol for manipulating text - 
colors, moving, deleting. In some cases, especially 
during fast-paced operations, 2 or 3 days may elapse 
before the “right” thing happens to a block of text. 
However, this can be compensated for during Rolloff 
Matrix processing. 

The Rolloff Matrix is essentially a collection of index tabs, 
so brevity is critical. Description and Disposition should be 
kept to 1-3 lines, 5 lines maximum. Start and stop dates tell 
the reader which DRs and/or console logs to examine for 
details, and of course the DR entries may point to specific 
log entries. Significant Events and Comments typically 
appear just once in the DR, so “Date Closed” for these is 
usually blank. 

Formally speaking, “Facility” refers to a rack or collection 
of racks onboard ISS that hosts equipment for specific types 
of scientific investigations, and “Payload” refers to a 
specific experiment, whether the experiment uses a Facility 
or not. In addition to these meanings, the columns serve to 
group non-payload items into major categories and sub- 
categories. 

In developing the Rolloff Matrix concept, there was much 


discussion as to when items should be transferred from the 
DR to the Matrix - when they start vs. when they’re 
finished, as part of the daily console-keeping routine vs. 
offline by an off-duty console operator, etc. Greater 
consistency has been achieved by updating or “auditing” the 
Matrix offline, processing several DRs in one sitting. This 
allows time and a relatively unstressed environment step 
back from the operations tempo, compare successive reports 
for progression and consistency, discern the essence of each 
given entry, and clean up the wording and/or Start/Stop 
dates. It takes about 2 to 3 hours to process a month’s worth 
of DRs. 

If one has a situation with round-the-clock staffing and a 
reasonable amount of low-pressure time on at least one 
shift, maintaining a Rolloff Matrix might be feasible on- 
console, but this is not workable for PAYCOM, and there’s 
much to be said for the “let it soak in” phenomenon intrinsic 
to processing the Matrix offline. If less than a month’s 
latency is desired/reports could be processed weekly. 

As with many endeavours, the Rolloff Matrix isn’t perfect, 
but therein lies potential for raising questions that can 
improve operations. While populating the matrix, it’s useful 
to include comments about what’s in the matrix in red and 
within brackets, [ like this ]. Such comments might flag 
data inconsistencies, suggest followup activity, or state 
insights gained while reviewing the information. 

A recently conceived idea (October, 2007) is to allow and 
encourage off-duty console operators to send comments or 
observations (based on reading the daily console log or on 
meetings relevant to current ops) to the on-duty operator for 
inclusion in the DR (labeled as an outside comment), and 
eventual transfer to the Rolloff Matrix. If practiced in 
moderation, this could enhance corporate consciousness. 

Figure 3 summarizes how the Console Log, Daily Reports, 
and Rolloff Matrix inter-relate. 
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Console Log (Filemaker) Daily Reports (Word) Rolloff Matrix (Excel) 

- - 1 2,500 Entries per year - 260 Reports per year - 250 Categorized Entries per year 

Can select records via text search 



E-mail comment from ort-dnty operator and would be partitioned only if forc ed by file size or application 

(Fresh log emrks distributed to team daily) performance limitations. So tar, -.his hasr.'i 'Happened. 


Figure 3 - Relationships Among Products 
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Conclusions 

Including Actions and Reminders in a mandatory Daily 
Report ensures that: 

Console operators view the information daily. 

Ops management personnel receiving the report are 
aware of items being tracked, and can compare them to 
their notes to eliminate duplication of effort and/or fill 
in gaps appropriate to the function of the reporting 
console position. 

Transferring Actions, Reminders, Significant Events, and 
Comments from the Daily Report (DR) to a Rolloff Matrix 
provides a categorized synopsis of issues and events (good, 
bad, and indifferent), that have risen above the noise level. 
The transfer is best accomplished in an offline mode, e.g., 
by an off-duty console operator processing a week or a 
month of Daily Reports in one sitting of 30 minutes to 3 
hours. Visual layout and color-coding within the Daily 
Report make preparing the report and transferring data to 
the matrix easier. 

Information contained in the Rolloff Matrix has several 
uses: 

Prepare to operate a given payload. Check its history 
for trends, issues, and/or “gotchas”. Start/stop dates 
and/or console log references (in Matrix or Daily 
Report entries) provide quick navigation to supporting 
details. 

At the beginning of a payload increment, determine 
which Reminders to continue, discontinue, or resurrect. 

At the end of an increment, identify lessons learned. 
(Two mechanisms are at work here. If a lesson is 
recognized at the time something happens, it can be put 
in the DR and hence the matrix, for ready recall later, as 
opposed to getting buried so deep in a console log that 
it’s not recalled months later. If a lesson is not noticed 
in the moment, it may be distilled in the process of 
reviewing all the items that rose above the noise.) 

Identify topics for “go-back” discussions and/or further 
research. 

Demonstrate thorough accountability and/or traceability 
of action items not covered in formal databases. 

Gain insight about and improve consistency of console 
operations themselves. Examples of things to look at: 
Types of actions tracked, and how long they’re 
typically carried; When an item is complete as opposed 
to when it’s actually removed from the DR; 
Consistency of what different operators include in the 
DR. 


means of capturing actions and “above the noise” issues that 
are not covered by primary ops databases, first because they 
are actions or are above the noise in their own right, and just 
as importantly because trends may develop over time that 
provide useful insight. Performing the capture via a report 
viewed by others enables feedback; validation, and 
reduction of gaps and/or overlaps, Routine harvesting, such 
as a rolloff matrix, summarizes and indexes the actions and 
issues so that they are not lost in the abyss of a 
comprehensive console log or hidden in reams of daily 
reports. 
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Generically, post-ops and/or in-ops analysis for long-term 
operations can be significantly enhanced by having a daily 
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